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In response to the Notification of Non-Compliant Appeal Brief mailed November 29, 
2007, the following is submitted. 

I. Real Party in Interest 

The inventors Christian ENSEL and Volkmar STERZING assigned all rights in the 
subject application to SIEMENS AKTIENGESELLSCHAFT on January 18, 2002 according to the 
Assignment executed January 18, 2002 by Christian ENSEL and January 25, 2002 by Volkmar 
STERZING, and recorded at Reel 012580, Frame 0846 on February 14, 2002. Therefore, the 
real party in interest is SIEMENS AKTIENGESELLSCHAFT. 

II. Related Appeals and interferences 

There are no related appeals or interferences known to Appellant, Appellant's legal 
representatives or the Assignee, SIEMENS AKTIENGESELLSCHAFT, which will directly affect 
or be directly affected by or have a bearing on the Board's decision in the pending appeal. 



III. Status of Claims 

Claims 12 and 15-26 are canceled and claims 1-11, 13, 14 and 27-29 are pending in the 
application. Claims 1-1 1, 13, 14 and 27-29 stand rejected under 35 U.S.C. § 103(a). The 
rejection of claims 1-11, 13, 14 and 27-29 is appealed. 

IV. Status of Amendments 

No Amendments were filed subsequent to the final Office Action. 

V. Summary of Claimed Subject Matter 

Independent Claim 1 

Claim 1 is directed to a method of monitoring a telecommunications network that has a 
plurality of devices and services capable of communication, as described in page 4, lines 13-16, 
of the substitute specification. 

The preamble of claim 1 recites a method for "computer-aided monitoring of a 
telecommunication network formed of devices capable of communication" (lines 1-2). An 
embodiment is shown in FIG, 4 and described in page 13, lines 11-12, and page 14, lines 13-19, 
of the substitute specification. 

The body of claim 1 recites "determining training activity parameters, each describing 
activity of at least one of a corresponding device and a corresponding service" at lines 4-5. 
Examples of "determining training activity parameters" are described in page 12, lines 22-13. In 
addition, activity parameters are defined in page 5, lines 14-15. Examples of activity parameters 
are described in page 7, lines 12-21 . 

Claim 1 also recites "determining possible dependences between devices and services 
from the training activity parameters" at lines 6-7. Examples of determining possible 
dependences are described in page 7, lines 8-16. 

Claim 1 further recites "determining from the possible dependences a normal range of 
dependence for at least some of the devices and services in essentially undisturbed states to 
train a neural network as a statistical estimator" at lines 8-11. An example of "determining from 
the possible dependences a normal range of dependence" is described in page 5, lines 14-15. 
One embodiment of M train[ing] a neural network as a statistical estimator 1 ' is described in page 8, 
lines 9-11. Examples of the "statistical estimator" are discussed in page 8, lines 22-24. 
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Lines 12-13 of claim 1 recite "determining current activity parameters, each describing 
activity of at (east one of a corresponding device and a corresponding service". An example of 
"determining current activity parameters" is discussed in page 14, lines 4-6. 

Lines 14-15 of claim 1 recite "comparing the current activity parameters by the statistical 
estimator with the normal range of dependence". An example of such a comparison is 
discussed in page 14, lines 13-14. 

Finally, claim 1 recites "determining from said comparing whether at least one of the 
devices and services in the telecommunication network has a communication performance 
different from the normal range of dependence in accordance with a predetermined criterion" in 
lines 16-18, An example is discussed in page 14, lines 13-19. 

Independent Claim 27 

Claim 27 is directed to a device for monitoring a telecommunications network that has a 
plurality of devices and services capable of communication, as described in page 4, lines 13-16, 
and page 9, lines 14-17, of the substitute specification. 

The preamble of claim 27 recites a "device for computer-aided monitoring of a telecom- 
munication network formed of devices capable of communication" (lines 1-2). An embodiment is 
described in page 9, lines 14-17, page 13, lines 11-12, and page 14, lines 13-19, of the 
substitute specification. 

The body of claim 27 recites "at least one processor to determine training activity 
parameters, each describing activity of at least one of a corresponding device and a 
corresponding service" at lines 3-4. As noted in page 9, lines 14-17, of the substitute 
specification, the application relates to both hardware and software. One skilled in the art would 
understand that the devices discussed in the specification may use more than one processor, 
because devices in a telecommunications network frequently use multiple processors. 
Furthermore, examples of determining training activity parameters are described in page 12, 
lines 22-23. Activity parameters are defined in page 5, lines 14-15 and examples of activity 
parameters are described in page 7, lines 12-21 . 

Claim 27 also recites "to determine possible dependences between devices and services 
from the training activity parameters" at lines 4-5. An examples of determining possible 
dependences are described in page 7, lines 8-16. One example is shown in FIG. 3 and 
described in page 13, lines 16-20, of the substitute specification, - , 
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Claim 27 further recites "to determine from the possible dependences a normal range of 
dependence for at least some of the devices and services in essentially undisturbed states to 
train a neural network as a statistical estimator" at lines 5-7. An example of "determining] from 
the possible dependences a normal range of dependence" is described in page 5, lines 14-15. 
One embodiment of "training] a neural network as a statistical estimator" is described in page 8, 
lines 9-1 1 . Examples of the "statistical estimator" are discussed in page 8, lines 22-24. 

Lines 8-9 of claim 27 recite "to determine current activity parameters, each describing 
activity of at least one of a corresponding device and a corresponding service". An example of 
"determin[ing] current activity parameters" is discussed in page 14, lines 4-6. 

Lines 9-10 of claim 27 recites "to compare the current activity parameters by the 
statistical estimator with the normal range of dependence". An example of such a comparison is 
discussed in page 14, lines 13-14. 

Finally, claim 27 recites "to determine from said comparing whether at least one of the 
devices and services in the telecommunication network has a communication performance 
different from the normal range of dependence in accordance with a predetermined criterion" in 
lines 10-13. An example is discussed in page 14, lines 13-19, 

Independent Claim 28 

Claim 28 is directed to a computer-readable storage medium storing at least one 
computer program monitoring a telecommunications network that has a plurality of devices and 
services capable of communication, as described in page 4, lines 13-16, and page 9, lines 14- 
15, of the substitute specification. 

The preamble of claim 28 recites 

at least one computer-readable storage medium storing at least 
one computer program for computer-aided monitoring of a 
telecommunication network formed of devices capable of 
communication, to control a processor to perform a method 

(lines 1-3). An example of the method stored on the computer-readable storage medium is 

shown in FiG. 4 and described in page 9, lines 14-15, page 13, lines 11-12, and page 14, lines 

13-19, of the substitute specification. 

The body of claim 28 recites "determining training activity parameters, each describing 
activity of at least one of a corresponding device and a corresponding service" at lines 5-6, 
Examples of "determining training activity parameters" are described in page 12, lines 22-23. In 
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addition, activity parameters are defined in page 5 T lines 14-15. Examples of activity parameters 
are described in page 7, lines 12-21. 

Claim 28 also recites "determining possible dependences between devices and services 
from the training activity parameters" at lines 7-8. Examples of determining possible 
dependences are described in page 7, lines 8-16. One example is described in page 13, lines 
16-20, of the substitute specification. 

Claim 28 further recites "determining from the possible dependences a normal range of 
dependence for at least some of the devices and services in essentially undisturbed states to 
train a neural network as a statistical estimator" at lines 9-1 1 . An example of "determining from 
the possible dependences a normal range of dependence" is described in page 5, lines 14-15. 
One embodiment of "training] a neural network as a statistical estimator" is described in page 8 t 
lines 9-1 1 . Examples of the "statistical estimator" are discussed in page 8, lines 22-24. 

Lines 12-13 of claim 28 recite "determining current activity parameters, each describing 
activity of at least one of a corresponding device and a corresponding service". An example of 
"determining current activity parameters" is discussed in page 14, lines 4-6. 

Lines 13-14 of claim 28 recite "comparing the current activity parameters by the statistical 
estimator with the normal range of dependence". An example of such a comparison is 
discussed in page 14, lines 13-14, 

Finally, claim 28 recites "determining from said comparing whether at least one of the 
devices and services in the telecommunication network has a communication performance 
different from the normal range of dependence in accordance with a predetermined criterion" in 
lines 15-17, An example is discussed in page 14, lines 13-19. 

Independent Claim 29 

Claim 29 is directed to a computer-readable storage medium storing at least one 
computer program monitoring a telecommunications network that has a plurality of devices and 
services capable of communication, as described in page 4 t lines 13-16, and page 9, lines 14- 
15, of the substitute specification. 

The preamble of claim 29 recites 

[a]t least one computer-readable storage medium storing at least 
one computer program for computer-aided training of a neural 
network as a statistical estimator for administering a 
telecommunication network formed of devices capable of 
communication, to control a processor to perform a method 
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(iines 1-4). An example of the method stored on the computer-readable storage medium is 
shown in FIG. 4 and described in page 9, lines 14-15 T page 13, lines 1 1-12, and page 14, lines 
13-19, of the substitute specification. 

The body of claim 29 recites "determining training activity parameters, each describing 
activity of at least one of a corresponding device and a corresponding service" at lines 5-6. 
Examples of "determining training activity parameters" are described in page 12, lines 22-23, 
and include a back-propagation training method. In addition, activity parameters are defined in 
page 5, lines 14-15. Examples of activity parameters are described in page 7, lines 12-21 

Claim 29 also recites "determining possible dependences between the devices and 
services from the training activity parameters" at lines 7-8. Examples of determining possible 
dependences are described jn page 7, lines 8-16. One example is shown in FIG. 3 and 
described in page 13, lines 16-20, of the substitute specification. 

Claim 29 further recites 

determining from the possible dependences a normal range of 
dependence for at least some of the devices and services in 
essentially undisturbed states to train the statistical estimator for 
determining current activity parameters, each describing activity of 
at least one of a corresponding device and a corresponding 
service and comparing the current activity parameters with the 
normal range of dependence 

(lines 9-13). An example of "determining from the possible dependences a normal range of 

dependence" is described in page 5, lines 14-15. One embodiment of "training a neural network 

as a statistical estimator" is described in page 8, lines 9-11. Examples of the "statistical 

estimator" are discussed in page 8, iines 22-24. An example of "determining current activity 

parameters" is discussed in page 14, lines 4-6. An example of "comparing the current activity 

parameters with the normal range of dependence" is discussed in page 14, iines 13-14. 

VI. Grounds of Rejection to be Reviewed on Appeal 

In the final Office Action mailed April 3, 2007, claims 1-1 1 , 13, 14, and 27-29 were 
rejected under 35 USC § 103(a) as unpatentable over Waclawskv et ai. (U.S. Patent No. 
5,974,457) in view of Nuansri et al. ("An Application of Neural Network and Rule-Based System 
for Network Management: Application Level Problems"). 

VII. Argument 

Numbered paragraph 5, on pages 2 of the final Office Action, states that claims "1-1 1 , 
13, 14, and 27-29 remain rejected under 35 USC § 103(a)" as unpatentable over Waclawskv et 
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aj. (U.S. Patent Number 5,974,457) in view of Nuansri et al. ("An Application of Neural Network 
and Rule-Based System for Network Management: Application Level Problems"). Nothing 
further is stated in the final Office Action regarding the rejection under 35 U.S.C. § 103(a). 
Therefore, in the argument that follows, the Applicants assume the Examiner has maintained the 
rejection of claims 1-11, 13, 14, and 27-29 as it appeared in the Office Action mailed October 21 , 
2006. 

Discussion of the prior art and one skilled in the art, 

Any discussion of "neural networks", "rule-based systems" and "expert systems", as 
described in the prior art of record, invariably involves a discussion of Artificial Intelligence. As 
described by "Artificial Intelligence: An Overview" (HonavarV., Principles of Artificial Intelligence, 
Com S 572, Fall 2006, Iowa State University, last revised August 21, 2006, hereinafter Honavar 
'06) , "[t]he term Artificial Intelligence refers to the enterprise of understanding and building 
intelligent systems" (see Exhibit 1). Honavar '06 then goes on to list milestones in the field and 
gives a brief discussion of "neural networks" (page 6 7 11 th bullet point), "rule-based systems" 
(page, 6, 2 nd bullet point) and "expert systems" (page 7, 7 th bullet point). Although Honavar '06 
was last revised in 2006, the definition provided by Honavar '06 and the subsequent discussion 
of Artificial Intelligence demonstrates that terms "neural networks", "rule-based systems" and 
"expert systems" as used in the prior art pertain to Artificial Intelligence. 

While both "rule-based systems" and "expert systems" (often referred together as 
"symbolic Artificial Intelligence") and "neural networks" relate to the field of Artificial Intelligence, 
those skilled in the art have long recognized that there is a difference in the two approaches to 
the problem of Artificial Intelligence. For example, in "Symbolic Artificial Intelligence and 
Numeric Artificial Neural Networks: Toward a Resolution of the Dichotomy" (Honavar V. r 
Computational Architectures Integrating Symbolic and Neural Processes, Sun, R, & Bookman, L 
(Ed.) New York, NY: Kluwer, 1994, hereinafter Honavar '94 , see Exhibit 2), Honavar '94 states 
that 

[i]t is often suggested that two major approaches have emerged - 
symbolic artificial intelligence (SAi) and (numeric) artificial 
intelligence (NANN or connectionist networks) and some (Norman 
1986 T Schneider, 1987) have even suggested that they are 
fundamentally and irreconcilably different 

(page 2, lines 1-5), While Honavar '94 later states "[t]his chapter argues that the dichotomy 

between SAI and NANN is more perceived than real" (page 2, Iines19~20), Honavar '94 

demonstrates that a person of ordinary skill in the art of Artificial Intelligence understands that 

there is an articulable difference between "rule-based systems" and "neural networks". 
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Of the prior art relied on in the Office Action mailed October 21 , 2006, Waclawskv et al. 
disclosed a only a rule-based system. Rule-based systems consist of a number of manually 
defined rules representing what the rule-based system's designers consider to be an expert in 
the field's knowledge base. Ruie-based systems are also referred to as "expert systems" or 
"knowledge-based systems". Another way of describing how a rule-based system is created and 
operates is: "[knowledge engineering is a field within artificial intelligence that develops 
knowledge-based systems. ... A major form of knowledge-based system[s] is an expert system, 
one designed to emulate the reasoning processes of an expert practitioner" (see 
"www.epistemics.co.uk/Notes/61 -0-0.htm", accessed August 29, 2007; as Exhibit 3; see also 
Honavar '06 , page, 6, 2 nd bullet point). Waclawsky et al. describes the embodiment of this 
process used in FIG. 1A-1 as "[t]he expert system analysis module 160 contains the rule based 
criteria modules 150, 150' and 150["], which contain rules which wili characterize the classes of 
traffic on the network being monitored" in column 4, lines 43-46. 

Nuansri et al. disclosed a hybrid system that puts 3 neural network system in a serial 
connection with a rule-based system. The two systems are discrete parts of the overall system, 
as shown in Fig. 13 on page 480. Specifically, in Nuansri et al. , the neural network system is 
labeled BRAINNE in Fig. 13 and the rule-based system is labeled NEXPERT according to 
section 5, Furthermore, as further described in section 5, "BRAiNNE is used as an automated 
knowledge acquisition tool that allows us to extract DNS faults (errors) and their causes while 
the NEXPERT system is later used as a rule-based expert system for the analysis and diagnosis 
part." Thus, the system described in Nuansri et al. used a neural network system as a pre- 
processor to a rule-based system. 

Patentable distinctions between the claims and prior art of record 

Claim 1 recites "determining from the possible dependences a normal range of 
dependence for at least some of the devices and services essentially undisturbed states to train 
a neural network as a statistical estimator" (claim 1 , lines 8-10). The first two lines of page 4 in 
the Office Action mailed October 21 , 2006 alleged that Nuansri et al. "train[ed] a neural network 
as a statistical estimator to effectuate network monitoring and diagnosing." The Applicant's 
disagree with this assessment of Nuansri et al. 

One shortcoming of the October 21 , 2006 Office Action's assessment regarding Nuansri 
et al is the failure to cite where Nuansri et al. disclosed a statistical estimator. Nothing can be 
found in Nuansri et al, that teaches any statistical methods used to act as a statistical estimator. 
What has been found, in sections 5.1 , describe the training of BRAINNE by reading a iog file to 
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retrieve error messages as one component and "extracting knowledge from experience and 
human experts and from forcing faults on a name server" to retrieve possible causes of the error 
messages as the second component. Since the training of BRAINNE uses log files to retrieve 
error messages, it is submitted that Nuansri et al. does not describe anything "determining from 
the possible dependences a normal range of dependence for at least some of the devices and 
services essentially undisturbed states" as recited in claim 1 at lines 8-9. Moreover, a statistical 
estimator, such as the neural network recited in claim 1 7 does not require an expert to create 
rules. Rather, the statistical estimator represents the required knowledge after training in implicit 
form. Thus, by relying on a rule based system, Waclawsky et al. teaches away from a statistical 
estimator training system. 

Additionally, as discussed above, Nuansri et al. does not describe the neural network 
system as a monitoring or diagnosing system, but rather as a pre-processor. This is evident by 
the structure shown in Fig. 13, where the neural network system (BRAINNE) outputs to the rule- 
based system (NEXPERT KB) in the sub-graph labeled "Learning Process" and the rule-based 
system (NEXPERT interface) received its input from a log file in the sub-graph labeled 
"Monitoring and diagnosing Process" in Fig. 13. 

In the final Office Action mailed April 3, 2007, the Examiner argues that Waclawsky et al. 
n has been cited in the rejection for detail in determining a norma! range of dependence" 
(numbered paragraph 8 r line 4). It is not clear where the Examiner has previously analyzed and 
demonstrated that Waclawsky et al. teaches or suggests "determining a normal range of depen- 
dence" and nothing has been found in the record supporting this statement in the final Office 
Action, instead, the October 21 , 2006 Office Action merely cites both Waclawsky et al. (column 
4, line 50 through column 5, line 4) and Nuansri etaj (page 478, first paragraph) as teaching or 
suggesting "determining from the possible dependences a normal range of dependence for at 
least some of the devices and services in essentially undisturbed states to train a neural network 
as a statistical estimator" (last four lines of page 4 of the October 26, 2007 Office Action). 

Waclawsky et al. states "[t]he comparison is performed by the rules contained in the rule 
based criteria modules 150 ... standards can be predetermined, predefined standards such as 
average utilization for particular types of traffic such as batch traffic, interactive traffic, voice 
traffic or video traffic" in column 4, lines 60-63, emphasis added. Thus, the "standard" as taught 
by Waclawsky et al. is a scalar value, specifically an "average" value. Waclawsky et al. then 
goes on to state "[a]nother important type of standard is the benchmark data set which is the 
accumulated history of behavior of traffic on the network" at column 4 t lines 63-65. Waclawsky 
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et al. does not further describe how the "benchmark data set" could be used as a "normal range 
of dependence" as recited in the claims (e.g. claim 1 at line 8, emphasis added), instead, 
Waclawsky et al. simply states that the "benchmark data sets ... progressively accumulate a 
more accurate representation of the expected behavior for the network and that standard can be 
substituted for the predetermined standard used by the rules in the rule based criteria modules" 
in column 4, line 66 to column 5, line 4). Given what has been quoted above from Waclawsky et 
aL r one of ordinary skill in the art would understand Waclawsky et al to teach or suggest using 
"benchmark data sets" to calculate an average and using the calculated average of the 
"benchmark data sets" as the standard instead of the "average utilization for particular types of 
traffic such as batch traffic, interactive traffic, voice traffic or video traffic" as the standard. 

For the foregoing reasons, it is submitted that the rejections of claims 1-1 1 , 13 T 14, and 
27-29 under 35 (JSC § 1 03(a) as unpatentable over Waclawsky et al. in view of Nuansri et al. 
was improper because nothing on record has been shown to teach or suggest a "statistical 
estimator" or a "normal range of dependence" as quoted gbove from the claims. The Board is 
respectfully requested to reverse the Examiner's final rejection. 

Improper Combination 

Furthermore, Applicants respectfully submit that the combination of Waclawsky et al. and 
Nuansri et al. is improper. In the October 21 , 2006 Office Action, Nuansri et al. is relied upon as 
disclosing the statistical estimator requirements of the claim as recited in the third operation of 
claim 1. However, all other operations recited in the claimed method are allegedly taught or 
suggested by Waclawsky et al. The references Nuansri et al. and Waclawsky et al. are 
disparate teachings raising the question of why a person skilled in the art would even consider 
these references for combination, a question the Board must answer 1 since the Examiner did 
not. The statements in the October 21 , 2006 Office Action that Nuansri et al. is "an analogous 
art" 2 and that "it would have been obvious to a person skilled in the art at the time of the 
applicant's invention to modify ..." 3 are conclusory statements that do not constitute "sufficient 
factual findings." 4 Applicants respectfully traversed the obviousness rejection based on 



1 See In re Lee, 277 F3d 1338, 61 USPQ2d 1430, 1434 (Fed. Cir. 2002), "the Board must explain the 
reasons one of ordinary skill in the art would have been motivated to select the references." 

2 See item 9 of the Office Action, lines 2-3. 

3 See item 10 of the Office Action, lines 7-8. 

4 MPEP 2144.08 II! states that ,r [e]xplicit findings on motivation or suggestion to select the claimed 
invention should also be articulated in order to support a 35 U.S.C. § 103 ground of rejection. . . 
Conclusory statements of similarity or motivation, without any articulated rational or evidentiary support, do 



10 



Waclawsky et al. and Nuansri et al. because there is insufficient evidence for a motivation to use 
the method of Nuansri et aL in the system described by Waclawsky et al. in the Response filed 
January 31 7 2007. While the required evidence of motivation to combine need not come from 
the applied references themselves, the evidence must come from somewhere within the record. 5 
In this case, the record fails to support the proposed combination. 

Dependent claims 2-14 depend from claim 1. Independent claims 27-29 recite statistical 
estimator limitations in a manner similar to claim 1 Thus, claims 2-14 and 27-29 distinguish 
over the applied art for the reasons discussed in regard to claim 1. 

Summary of Arguments 

For the reasons set forth above and in the Request for Reconsideration filed January 31, 
2007, it is respectfully submitted that claims 1-11, 13, 14, and 27-29 patentably distinguish over 
the prior art. Thus, it is respectfully submitted that the Examiner's final rejection of the claims is 
without support and, therefore, erroneous. Accordingly, the Board of Patent Appeals and 
Interferences is respectfully urged to so find and to reverse the Examiner's final rejection. 

Please charge the required fee in the amount of $500.00 to our Deposit Account No. 19- 
3935. If any additional fees are required for this Appeal Brief, please charge same to our 
Deposit Account No. 19-3935. 

Respectfully submitted, 
STAAS & HALSEY LLP 

Date: By: 

David E. Moore 
Registration No. 59,047 

1201 New York Avenue, NW, Suite 700 
Washington, D.C. 20005 
Telephone: (202)434-1500 
Facsimile: (202)434-1501 



not constitute sufficient factual findings," 

5 In re Lee, 277 R3d 1338, 1343-4, 61 USPQ2d 1430 {Fed. Cir. 2002) ("The factual inquiry whether to 
combine references ... must be based on objective evidence of record, ... [The] factual question of 
motivation ... cannot be resolved on subjective belief and unknown ajjihqrity. ... Thus, the Board must 
not only assure that the requisite findings are made, based on evidence of record, but must also explain 
the reasoning by which the findings are deemed to support the agency's conclusion"). 



11 



Vill. Claims Appendix 

1 . A method for computer-aided monitoring of a telecommunication network formed of 
devices capable of communication, said method comprising: 

determining training activity parameters, each describing activity of at least one of a 
corresponding device and a corresponding service; 

determining possible dependences between devices and services from the training 
activity parameters; 

determining from the possible dependences a normal range of dependence for at least 
some of the devices and services in essentially undisturbed states to train a neural network as a 
statistical estimator; 

determining current activity parameters, each describing activity of at least one of a 
corresponding device and a corresponding service; 

comparing the current activity parameters by the statistical estimator with the normal 
range of dependence; and" 

determining from said comparing whether at least one of the devices and services in the 
telecommunication network has a communication performance different from the normal range 
of dependence in accordance with a predetermined criterion. 

2. The method as claimed in claim 1 , wherein at least some of the devices are 
constructed as terminals capable of communication, 

3. The method as claimed in claim 1 , wherein the training activity parameters are 
determined within a predetermined time interval. 

4. The method as claimed in claim 1 , 

wherein said determining of each training activity parameter is performed by the 
corresponding device, and 

wherein said method further comprises transmitting the training activity parameters to an 
administration unit which performs said comparing and determining based on said comparing, 

5. The method as claimed in claim 1 , wherein said determining of each training activity 
parameter is performed by a training activity parameter determining unit separate from the 
corresponding device. 
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6. The method as claimed in claim 1, further comprising determining communication- 
dependent dependences between at least some of the devices and services. 

7. The method as claimed in claim 1 r further comprising determining possible directional 
dependences with regard to directions of communication between at least some of the devices 
and services, 

8. The method as claimed in claim 1, 

further comprising determining data of at least some of the devices and services, and 
wherein said determining of the training activity parameters is based on the data. 

9. The method as claimed in claim 1, wherein said determining of the training activity 
parameters uses all possible pairs of the devices and pairs of services, 

10. The method as claimed in claim 9, further comprising: 

storing the training activity parameters determined from the pairs of devices in a matrix; 

and 

determining the normal range of dependence from a structure of the matrix. 

11 . The method as claimed in claim 1 , wherein at least one of the following parameters 
is determined as one of the training activity parameters 

data packets sent or received by the at [east one of a corresponding device and a 
corresponding service, 

processor utilization of the corresponding device, 
a number of predetermined system function calls, and 

existence of at least one of predetermined processes and predetermined computer 
programs. 

13. (Original) The method as claimed in claim 1 , further comprising generating an alarm 
signal when at least one device in the telecommunication network differs from the normal range 
of dependence in accordance with the predetermined criterion, 

14. (Original) The method as claimed in claim 1, further comprising at least one of 
determining a disturbance of one of the devices in the telecommunication network; 
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determining an unauthorized attempt to access one of the devices; and 
determining an unauthorized access attempt by one of the devices. 

27. A device for computer-aided monitoring of a telecommunication network formed of 
devices capable of communication, comprising: 

at least one processor to determine training activity parameters, each describing activity 
of at least one of a corresponding device and a corresponding service, to determine possible 
dependences between devices and services from the training activity parameters, to determine 
from the possible dependences a normal range of dependence for at least some of the devices 
and services in essentially undisturbed states to train a neural network as a statistical estimator, 
to determine current activity parameters, each describing activity of at least one of a 
corresponding device and a corresponding service, to compare the current activity parameters 
by the statistical estimator with the normal range of dependence, and to determine from said 
comparing whether at least one of the devices and services in the telecommunication network 
has a communication performance different from the normal range of dependence in 
accordance with a predetermined criterion. 

28. At least one computer-readable storage medium storing at least one computer 
program for computer-aided monitoring of a telecommunication network formed of devices 
capable of communication, to control a processor to perform a method comprising: 

determining training activity parameters, each describing activity of at least one of a 
corresponding device and a corresponding service; 

determining possible dependences between devices and services from the training 
activity parameters; 

determining from the possible dependences a normal range of dependence for at least 
some of the devices and services in essentially undisturbed states to train a neural network as a 
statistical estimator; 

determining current activity parameters, each describing activity of at least one of a 
corresponding device and a corresponding service; comparing the current activity parameters by 
the statistical estimator with the normal range of dependence; and 

determining from said comparing whether at least one of the devices and services in the 
telecommunication network has a communication performance different from the normal range 
of dependence in accordance with a predetermined criterion. 
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29. At least one computer-readable storage medium storing at least one computer 
program for computer-aided training of a neural network as a statistical estimator for 
administering a telecommunication network formed of devices capable of communication, to 
control a processor to perform a method comprising: 

determining training activity parameters, each describing activity of at least one of a 
corresponding device and a corresponding service; 

determining possible dependences between the devices and services from the training 
activity parameters; and 

determining from the possible dependences a normal range of dependence for at least 
some of the devices and services in essentially undisturbed states to train the statistical 
estimator for determining current activity parameters, each describing activity of at least one of a 
corresponding device and a -corresponding service and comparing the current activity 
parameters with the normal range of dependence. 
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Abstract 

This chapter reviews common-sense definitions of intelligence; motivates the research in artificial in- 
telligence (AI) that is aimed at design and analysis of programs .and computers that model minds/brains; 
lays out the fundamental guiding hypothesis of AI; reviews the historical' development of AI as a sci- 
entific and engineering discipline; explores the relationship of AI to other disciplines; and presents an 
overview of the scope, problems, and major areas of AI. Hopefully this discussion will not only provide a 
useful context for the technical material that follows but also convey a sense of what scientists, engineers, 
mathematicians, and philosophers who have been drawn to the field find exciting about AI. The views 
presented here are necessarily biased by the author's own research. The readers are encouraged to explore 
alternative views and perspectives on this subject. 
©Vasant Honavar, 1992-2006. 

1 What is Intelligence? 

Try precisely defining intelligence. It is next to impossible. Despite the wide use (and misuse) of terms 
such as intelligent systems^ there is no widely agreed-upon scientific definition of intelligence. It is therefore 
useful to think of intelligence in terms of an open collection of attributes. What follows is a wish-list of 
general characteristics of intelligence that contemporary researchers in AI and cognitive science are trying 
to understand and replicate. It is safe to say that no existing AI system comes anywhere close to exhibiting 
intelligence as characterized here except perhaps in extremely narrowly restricted domains (e.g., organic 
chemistry, medical diagnosis, information retrieval, network routing, military situation assessment, financial 
planning): 

• Perception — manipulation, integration, and interpretation of data provided by sensors (in the 
context of the internal state of the system — including purposeful, goal-directed, active perception). 

• Action — coordination, control, and use of effectors to accomplish a variety of tasks including explo- 
ration and manipulation of the environment, including design and construction of tools towards this 
end. 

• Reasoning — deductive (logical) inference, inductive inference, analogical inference — including rea- 
soning in the face of uncertainty and incomplete information, hypothetical reasoning, justification and 
-explanation of inferences, evaluation of explanations, adapting explanations in the light of falsified 
assumptions or changing world states. 

* Principles of Artificial Intelligence, Com S 572, Fall 2006, Iowa State University ©Vasant Honavar, 1992-2006 
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• Adaptation and Learning — adapting behaviour to better cope with changing environmental de- 
mands, discovery of regularities , explanation of observations in terras of known facts and hypotheses, 
construction of task-specific internal representations of the environment , discovery of procedures, learn- 
ing to differentiate despite similarities and generalize despite differences, learning to describe specific 
domains in terms of abstract theories and concepts, learning to use> adapt, and extend language, 
learning to reason, plan, and act, 

• Communication — with other intelligent agents including humans using signals, signs, icons, symbols, 
sound, pictures, touch, language and other communication media — including communication of goals, 
desires, beliefs, narratives of real and imaginary episodes, explanation of actions and events. 

• Planning and goal-directed problem-solving — Formulation of plans — sequences or agenda 
of actions to accomplish externally or internally determined goals, evaluating and choosing among 
alternative plans t adapting plans in the face of unexpected changes in the environment, explaining 
and justifying plans, modifying old plans to fit new tasks, handling complexity by abstraction and 
simplification. 

• Autonomy ■ — Setting, of goals, deciding on the appropriate course of actions to take in order to 
accomplish the goals or directives (without explicit instructions from another entity), executing the 
actions to satisfy the goals, adapting the actions and/or goals as necessary to deal with any unforeseen 
circumstances (to the extent permitted by the agent's physical capabilities and the environmental 
constraints). 

a Creativity — exploration, modification, and extension of domains (e.g., language, mathematics, mu- 
sic) by manipulation of domain-specific constraints, or by other means. 

• Reflection and awareness — of internal processes (e.g., reasoning, goals, etc.) of self as well as 
other agents. 

• Aesthetics — articulation and use of aesthetic principles. 

• Organization — into social groups based on shared objectives, development of shared conventions to 
facilitate orderly interaction, culture. 

Most people would probably agree that the hallmark of intelligence is almost certainly not simply the 
ability to display some or all of the listed attributes but doing so on a broad and open-ended (not precisely 
pre-specified) set of domains and under a broad range of (a-priori unknown, possibly context-dependent 
and domain-specific) constraints (e.g., time allowed, tools available, accuracy desired). It is also clear that 
different systems — be they natural or artificial — can display different subsets of the attributes of intelligence 
to differing degrees, 

2 What is Artificial Intelligence (AI)? 

The attempt to understand intelligence entails building theories and models of (appropriately embodied) 
brains and minds, both natural as well as artificial. Prom the earliest writings of India and Greece, this has 
been a central problem in philosophy. The advent of the digital computer in the 1950's made this a central 
concern of computer scientists as well. The parallel development of the theory of computation (by John von 
Neumann, Alan Turing, Emil Post, Alonzo Church, Stephen Kleene, Markov and others) provided a new set 
of tools with which to approach this problem — through analysis, design, and evaluation of computers and 
programs that exhibit aspects of intelligent behavior — such as the ability to recognize and classify patterns; 
to reason from premises to logical conclusions; and to learn from experience. 

The term Artificial Intelligence refers to the the enterprise of understanding and building intelligent 
systems, AI folklore credits John McCarthy {who incidentally, made several major contributions to AI and 
Computer Science in their infancy — by designing the programming language LISP and the first time-sharing 
operating system) with inventing the term during the workshop at Dartmouth in 1956 where the field took 
on its modern incarnation. 

Here are some descriptions of AL 
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• Descartes (1556-1650) discounts sensory experience as untrustworthy and justifies his own existence 
in terms of thought: Cogito ergo sum (I think, therefore I am) and with other contemporary thinkers, 
establishes the notion that the structure of ideas about the world are not necessarily the same as the 
structure of their subject matter, an idea which underlies much of the methodology of AI, epistemology, 
psychology, mathematics, and modern literature. 

• Hobbs (1650) proposes that thinking is a rule-based computational process analogous to arithmetic. 

• Leibnitz (1646-1716) seeks a general method in which all truths will be reduced to a kind of calculation. 

• Boole (1815-1864) puts forth his study of logic and probability as an investigation into the laws of 
thought. 

• Russell, Frege, Tarski (1910-1950) formalize logic and reduce large portions of mathematics to logic; 
Russell writes an influential book Princvpia Maihtmatica. Tarski introduces the theory of reference for 
relating objects in a logic to objects in the world, laying the foundations of formal semantics, 

• Hilbert (1862-1943) presents the decision problem . Is there an effective procedure for determining 
whether or not a given theorem logically follows from a given set of axioms? 

• Godel (1906-1978) shows the existence of an effective procedure to prove any theorem in Frege's logic 
and proves the incompleteness theorem 

• Turing (1912-1954) invents the luring Machine to formalize the notion of an effective procedure 

• Turing, Church, Kleene, Post (1930-50) Turing and Church put forth the Church-Turing thesis that 
Turing machines are universal computers. Kleene and Post propose other Turing-equivalent models of 
computation 

• Several special purpose analog and digital computers are built (including the Atanasofl-Berry Com- 
puter) 

• Chomsky (1956) develops the Chomsky hierarchy of languages formalize the common-sense notion of 
computation in terms of effective procedures and universal computers. 

• Rashevsky, McCulloch, Ashby, Rosenblatt (1930-60) — work on early neural network models. 

• Von Neumann, McCulloch (1940-1956), investigate the relationship between the brain and the 
computer 

• Von Neumann and Morgenstern (1946) develop a formal framework for rational decision making 
under uncertainty 

• Shannon (1948) develops information theory, laying the foundations of coding and communication 

• von Neumann (1956) works out a detailed design for a stored-program digital computer 

• Wiener, Lyapunov (1956) develop the science of cybernetics to study control and communication 
in humans and machines. 

• McCarthy, Minsky, Newell, Selfridge, Simon, Turing, Uhr, et aL (1956) establish AI in a 
workshop organized at Dartmouth College at the initiative of John McCarthy. 

• Several digital computers are constructed and universal languages for programming them are developed 
e.g., Lisp, Snobol, Fortran. 

• Wittgenstein (1950) challenges many of the underlying assumptions of the rationalist tradition, 
including the foundations of language, science, and knowledge itself; and proposes that the meaning of 
an utterance depends on being situated in a human, cultural content (e.g., the meaning of the word 
"chair" to me is dependent on my having a physical body that can take on a sitting posture and the 
cultural convention for using chairs). 
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• Dantzig and Edmunds (1960-62) introduce reduction a general transformation from one class of 
problems to another 

• Cobham and Edmunds (1964-65) introduce polynomial and exponential complexity 

• Cook and Karp (1971-72) develop the theory of NP-completeness which helps recognize problems 
that are intractable 

• Cerf (1974) invents the Internet 

• Husserl, Heidegger (1960-1975) articulate the view that abstractions must be rooted or grounded in 
the concrete lavensvelt or life-world i.e., the rationalist model of Aristotle is very much secondary to 
the concrete world that supported it; in the existentialist /phenomenological view, intelligence is not 
knowing what is true, but knowing how to cope with a world that is constantly changing. 

• 1960-1970 — untempered optimism fueled by early success on some problems thought to be hard (e.g., 
theorem proving) is tempered by slow progress on many problems thought to be easy (e.g., visual 
pattern recognition); much of the AI work is based in the rationalist/logical tradition; the field is 
fragmented into sub-areas focused on problem-solving, knowledge representation and inference, vision, 
planning, language processing, learning, etc. 

• 1970-mid 1980s — investigation of knowledge representation and reasoning leads to many practical 
tools such as expert systems; the difficulty task of knowledge. engineering draws attention to the need 
for systems capable of learning from experience and interaction with the world 

• Internet is rolled out in 1984 

• Mid 1980s- 1990 — some of the failures of rationalist/logical approaches to AI lead to renewed interest 
in biologically inspired neural network and evolutionary models which lead to modest successes on 
some problems (e.g., pattern recognition) prompting a few to prematurely proclaim the death of "good 
old fashioned AI". Some propose alternative approaches (in the Heideggerian tradition) while others 
discover (and rediscover) the limitations of the alternatives being proposed. 

• Mid 1980s-mid 1990s — progress in algorithmic models of learning begins to offer promising and practi- 
cal alternatives to knowledge engineering and AI technologies begin to be used in critical components of 
large software systems; The most successful approaches incorporate the elements of both the rational- 

. ist/logical/symbolic tradition and the existential/phenomenological/non-symbolic tradition; proposals 
for reconciling the two approaches begin to appear; maturing of the several subfields of AI such as vi- 
sion, language processing, knowledge representation, planning, etc. leads to insights on the capabilities 
as well as limitations of the techniques that were developed and redirects attention on the problem of 
building intelligent agents as opposed to subsystems and this further fuels synthesis of hybrid models. 

• Berners-Lee (1989-91) invents the world wide web 

• Mid 1990s-present — AI technologies continue to find applications in adaptive information retrieval, 
data mining and knowledge discovery from databases, customizable software systems, smart devices 
(e.g., homes, automobiles), agile manufacturing systems, autonomous vehicles, healthcare systems, 
medical informatics, etc. slow but steady progress on fundamental AI research problems continues. 
Synthesis of traditional logic-based systems, soft and adaptive computing technologies (e.g., neural 
networks, probabilistic models, etc.), crossdisciplinary work in cognitive sciences, and synthesis of soft- 
ware agents and multi-agent systems leads to the emergence of nouvelle AI which views intelligence 
as an emergent behavior resulting from interactions (e.g., communication, coordination, competition) 
among large numbers of autonomous or sem-autonomous entities (be they neurons, computer pro- 
grams, individuals) that are situated in the world, display structure and organization at multiple 
spatial and temporal scales, and interact with the world through sensors and effectors; a host of fun- 
damental problems such as design of individual agents, inter-agent communication and coordination, 
agent organizations, become topics of active research. 
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INTELLIGENCE AND NUMERIC 
ARTIFICIAL NEURAL NETWORKS: 
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1 INTRODUCTION 

The attempt to understand intelligence entails building theories and models of 
brains and minds, both natural as well as artificial- From the earliest writings 
oflndia and Greece, this has been a central problem in philosophy. The advent 
of the digital computer in the 1950*5 made this a central concern of computer 
scientists as well (Turing, 1950). The parallel development of the theory of 
computation (by John von Neumann, Alan Turing, Emil Post, Alonzo Church, 
Steven Kleene, Markov and others) provided a new set of tools with which 
to approach this problem — through analysis, design, and evaluation of com- 
puters and programs that exhibit aspects of intelligent behavior — such as the 
ability to recognize and classify patterns; to Teason from premises to logical 
conclusions; and to learn from experience. 

In their pursuit of artificial intelligence and mind/brain modelling > some wrote 
programs that they executed on serial stored-program computers (e.g., Newell, 
Shaw and Simon, 1963; Feigenbaum, 1963); Others had more parallel, brain- like 
networks of processors (reminiscent of today's connect ionist networks) in mind 
and wrote more or less precise specifications of what such a realisation of their 
programs might look like (e.g., Rashevsky, 1960; McCulloch and Pitts, 1943; 
Selfridge and Neisser, 1963; Uhr and Vossler, 1963); and a few took the middle 
ground (Uhr, 1973; Holland, 1975; Minsky, 1963; Arbib, 1972; Grossberg, 1982; 
Klir, 1985), 



1 



2 



Chapter 12 



It is often suggested that two major approaches have emerged ■ — symbolic 
artificial intelligence (SAI) and (numeric) artificial neural networks (NANN 
or connectionist networks) and some (Norman, 1986; Schneider, 1987) have 
even suggested that they are fundamentally and perhaps irreconcilably differ- 
ent. Indeed "it is this apparent dichotomy between the two apparently dis- 
parate approaches to modelling cognition and engineering intelligent systems 
that is responsible for the current interest in computational architectures for 
integrating neural and symbolic processes. This topic is the focus of several re- 
cent books .(Honavar and Uhr> 1994a; Goonatilake and Khebbal, 1994; Levine 
and Aparicioiv, 1994; Sun and Bookman, 1994)/ This raises some important 
questions: What exactly are symbolic processes? What do they have to do 
with SAI? What exactly are neural processes? What do they have to do with 
NANN? What (if anything) do SAI and NANN have in common? How (if at 
all) do they differ? What exactly are computational architectures? Do SAI and 
NANN paradigms need to be integrated? Assuming that the answer to the last 
question is yes, what are some possible ways one can go about designing com- 
putational architectures for this task? This chapter is an attempt to explore 
some of these fundamental questions in some detail. 

This chapter argues that the dichotomy between SAI and NANN is more per- 
ceived than real. So our problems lie first in dispelling misinformed and wrong 
notions, and second (perhaps more difficult) in developing systems that take ad- 
vantage of both paradigms to build useful theories and models of minds/brains 
on the one hand, and robust, versatile and adaptive intelligent systems on the 
other. The first of these problems is best addressed by a critical examination of 
the popular conceptions of SAI and NANN systems along with their philosoph- 
ical and theoretical foundations as well as their practical implementations; and 
the second by a judicious theoretical and experimental exploration of the rich 
and interesting space of designs for intelligent systems that integrate concepts, 
constructs, techniques and technologies drawn from not only SAI (Ginsberg, 
1993; Winston, 1992) and NANN (McClelland and Rumelhart, 1986; Kung, 
1993; Haykin, 1994; Zeidenberg, 1989), but also other related paradigms such 
as statistical and syntactic pattern recognition (Dud a and Hart, 1973; Fuku- 
naga, 1990; Fu, 1982; Miclet, 1986)), control theory (Narendra and Annaswamy, 
1989) systems theory (Klir, 1969), genetic algorithms (Holland, 1975; Goldberg, 
1989; Michalewicz, 1992) and evolutionary programming (Koza, 1992). Explo- 
ration of such designs should cover a broad range of problems in perception, 
knowledge representation and inference, robotics, language, and learning, and 
ultimately, integrated systems that display what might be considered human- 
like general intelligence. 
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Knowledge Engineering 



Knowledge engineering is a field within artificial intelligence that develops 
knowledge-based systems. Such systems are computer programs that contain 
large amounts of knowledge, rules and reasoning mechanisms to provide solutions 
to real-world problems. 

A major form of knowledge-based system is an expert system, one designed to 
emulate the reasoning processes of an expert practitioner (i.e. one having 
performed in a professional role for very many years). Typical examples of expert 
systems include diagnosis of bacterial Infections, advice on mineral exploration 
and assessment of electronic circuit designs. 



Importance of Knowledge Acquisition 

The early years of knowledge engineering were dogged by problems. Knowledge 
engineers found that acquiring enough high-quality knowledge to build a robust 
and useful system was a very long and expensive activity. As such, knowledge 
acquisition was identified as the bottleneck in building an expert system. This led 
to knowledge acquisition becoming a major research field within knowledge 
engineering. 

The aim of knowledge acquisition Is to develop methods and tools that make the 
arduous task of capturing and validating an expert's knowledge as efficient and 
effective as possible. Experts tend to be important and busy people; hence It is 
vital that the methods used minimise the time each expert spends off the job 
taking part in knowledge acquisition sessions. 



Knowledge Engineering Principles 

Since the mid-1980s, knowledge engineers have developed a number of principles, 
methods and tools that have considerably improved the process of knowledge 
acquisition. Some of the key principles are summarised as follows: 

• Knowledge engineers acknowledge that there are different types of 
knowledge, and that the right approach and technique should be used for 
the knowledge required. 

• Knowledge engineers acknowledge that there are different types of experts 
and expertise, such that methods should be chosen appropriately. 

• Knowledge engineers recognise that there are different ways of 
representing knowledge, which can aid the acquisition, validation and re-use 
of knowledge. 

• Knowledge engineers recognise that there are different ways of using 
knowledge, so that the acquisition process can be guided by the project 
aims. 

• Knowledge engineers use structured methods to increase the efficiency of 
the acquisition process. 



Knowledge Engineering Methodologies 

Epistemics is involved in three methodologies to support the development of 
knowledge systems: 
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